Statistical Models for the Analysis of Optimization Algorithms With Benchmark Functions

Frequentist statistical methods, such as hypothesis testing, are standard practice in papers that provide benchmark comparisons. Unfortunately, these methods have often been misused, e.g., without testing for their test assumptions or controlling family-wise errors multiple group comparisons, among several other problems. Bayesian Data Analysis (BDA) addresses many of the previously mentioned shortcomings but its use is not widely spread analysis empirical data evolutionary computing community. This paper provides three main contributions. First, we motivate need utilizing and an overview this topic. Second, discuss practical aspects BDA to ensure our models valid results transparent. Finally, five can be used answer research questions. The online appendix a step-by-step guide on how perform discussed paper, including code models, transformations tables figures.